Linear and Order Statistics Combiners for Pattern Classification

نویسندگان

  • Kagan Tumer
  • Joydeep Ghosh
چکیده

Several researchers have experimentally shown that substantial improvements can be obtained in difficult pattern recognition problems by combining or integrating the outputs of multiple classifiers. This chapter provides an analytical framework to quantify the improvements in classification results due to combining. The results apply to both linear combiners and order statistics combiners. We first show that to a first order approximation, the error rate obtained over and above the Bayes error rate, is directly proportional to the variance of the actual decision boundaries around the Bayes optimum boundary. Combining classifiers in output space reduces this variance, and hence reduces the ”added” error. If N unbiased classifiers are combined by simple averaging, the added error rate can be reduced by a factor of N if the individual errors in approximating the decision boundaries are uncorrelated. Expressions are then derived for linear combiners which are biased or correlated, and the effect of output correlations on ensemble performance is quantified. For order statistics based non-linear combiners, we derive expressions that indicate how much the median, the maximum and in general the ith order statistic can improve classifier performance. The analysis presented here facilitates the understanding of the relationships among error rates, classifier boundary distributions, and combining in output space. Experimental results on several public domain data sets are provided to illustrate the benefits of combining and to support the analytical results.

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Theoretical Foundations of Linear and Order Statistics Combiners for Neural Pattern Classifiers

Several researchers have experimentally shown that substantial improvements can be obtained in diicult pattern recognition problems by combining or integrating the outputs of multiple classiiers. This paper provides an analytical framework to quantify the improvements in classiication results due to combining. The results apply to both linear combiners and the order statistics combiners introdu...

متن کامل

Robust Combining of Disparate Classi ers through Order Statistics

Integrating the outputs of multiple classiiers via combiners or meta-learners has led to substantial improvements in several diicult pattern recognition problems. In this article we investigate a family of combiners based on order statistics, for robust handling of situations where there are large discrepancies in performance of individual classiiers. Based on a mathematical modeling of how the...

متن کامل

Analysis of Linear and Order Statistics Combiners for Fusion of Imbalanced Classifiers

So far few theoretical works investigated the conditions under which specific fusion rules can work well, and a unifying framework for comparing rules of different complexity is clearly beyond the state of the art. A clear theoretical comparison is lacking even if one focuses on specific classes of combiners (e.g., linear combiners). In this paper, we theoretically compare simple and weighted a...

متن کامل

Classiier Combining through Trimmed Means and Order Statistics

| Combining the outputs of multiple neural networks has led to substantial improvements in several dii-cult pattern recognition problems. In this article, we introduce and investigate robust combiners, a family of classiiers based on order statistics. We focus our study to the analysis of the decision boundaries, and how these boundaries are aaected by order statistics combiners. In particular,...

متن کامل

Fixed and Trained Combiners for Fusion of Imbalanced Pattern Classifiers

In the past decade, several rules for fusion of pattern classifiers’ outputs have been proposed. Although imbalanced classifiers, that is, classifiers exhibiting very different accuracy, are used in many practical applications (e.g., multimodal biometrics for personal identity verification), the conditions of classifiers’ imbalance under which a given rule can significantly outperform another o...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

عنوان ژورنال:
  • CoRR

دوره cs.NE/9905012  شماره 

صفحات  -

تاریخ انتشار 1999